Monitoring system, monitoring method, and monitoring program

ABSTRACT

An abnormality behind a shielding object is discovered. There is provided a monitoring system including a video acquirer, a detector, and a notifier. The video acquirer of the monitoring system acquires a video. The detector of the monitoring system detects entering of a target object into a blind spot generated by a shielding object in the video and appearance of the target object from the blind spot. If the target object does not appear from the blind spot even after a first time elapses since entering of the target object into the blind spot, the notifier of the monitoring system makes a notification.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.15/559,526, filed Sep. 19, 2017, which is a National State ofInternational Application No. PCT/JP2015/083912 filed Dec. 2, 2015,claiming priority based on Japanese patent application No. 2015-058279filed on Mar. 20, 2015, the contents of all of which are incorporatedherein by reference in their entirety.

TECHNICAL FIELD

The present invention relates to a monitoring system, a monitoringmethod, and a monitoring program.

BACKGROUND ART

In the above technical field, patent literature 1 discloses a techniquein which if a tracking object is hidden by a shielding object andappears again, the same identification number is added to the trackingobject. Patent literature 2 discloses a technique in which if a trackingvehicle is hidden by a shielding object, the predicted position of thetracking vehicle is displayed on a monitor. Furthermore, patentliterature 3 discloses a technique of giving an alarm by predicting therisk that an object in a blind spot of a driver runs out.

CITATION LIST Patent Literature

Patent literature 1: Japanese Patent Laid-Open No. 2007-334631

Patent literature 2: Japanese Patent Laid-Open No. 2009-193135

Patent literature 3: Japanese Patent Laid-Open No. 2013-97480

SUMMARY OF THE INVENTION Technical Problem

In the techniques described in the above patent literatures, however, itis impossible to discover an abnormality behind a shielding object.

The present invention enables to provide a technique of solving theabove-described problem.

Solution to Problem

One aspect of the present invention provides a monitoring systemcomprising:

a video acquirer that acquires a video;

a detector that detects entering of a target object into a blind spotgenerated by a shielding object in the video and appearance of thetarget object from the blind spot; and

a notifier that makes a notification if the target object does notappear from the blind spot even after a first time elapses sinceentering of the target object into the blind spot.

Another aspect of the present invention provides a monitoring methodcomprising:

acquiring a video;

detecting entering of a target object into a blind spot generated by ashielding object in the video and appearance of the target object fromthe blind spot; and

making a notification if the target object does not appear from theblind spot even after a first time elapses since entering of the targetobject into the blind spot.

Still other aspect of the present invention provides a monitoringprogram for causing a computer to execute a method, comprising:

acquiring a video;

detecting entering of a target object into a blind spot generated by ashielding object in the video and appearance of the target object fromthe blind spot; and

making a notification if the target object does not appear from theblind spot even after a first time elapses since entering of the targetobject into the blind spot.

Advantageous Effects of Invention

According to the present invention, it is possible to discover anabnormality behind a shielding object.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram showing the arrangement of a monitoring systemaccording to the first example embodiment of the present invention;

FIG. 2 is a view for explaining an outline of the operation of amonitoring system according to the second example embodiment of thepresent invention;

FIG. 3 is a block diagram showing the arrangement of the monitoringsystem according to the second example embodiment of the presentinvention;

FIG. 4A is a table showing the structure of a prediction table providedin the monitoring system according to the second example embodiment ofthe present invention;

FIG. 4B is a table showing the structure of a shielding object tableprovided in the monitoring system according to the second exampleembodiment of the present invention;

FIG. 4C is a table showing the structure of a notification tableprovided in the monitoring system according to the second exampleembodiment of the present invention;

FIG. 5 is a view showing the hardware arrangement of the monitoringsystem according to the second example embodiment of the presentinvention;

FIG. 6 is a flowchart illustrating the processing procedure of themonitoring system according to the second example embodiment of thepresent invention;

FIG. 7 is a view for explaining an outline of the operation of amonitoring system according to the third example embodiment of thepresent invention;

FIG. 8 is a block diagram showing the arrangement of the monitoringsystem according to the third example embodiment of the presentinvention;

FIG. 9 is a flowchart illustrating the processing procedure of themonitoring system according to the third example embodiment of thepresent invention;

FIG. 10 is a view for explaining an outline of the operation of amonitoring system according to the fourth example embodiment of thepresent invention;

FIG. 11 is a block diagram showing the arrangement of the monitoringsystem according to the fourth example embodiment of the presentinvention;

FIG. 12 is a flowchart illustrating the processing procedure of themonitoring system according to the fourth example embodiment of thepresent invention;

FIG. 13 is a view for explaining an outline of the operation of amonitoring system according to the fifth example embodiment of thepresent invention;

FIG. 14 is a block diagram showing the arrangement of the monitoringsystem according to the fifth example embodiment of the presentinvention; and

FIG. 15 is a flowchart illustrating the processing procedure of themonitoring system according to the fifth example embodiment of thepresent invention.

DESCRIPTION OF EXAMPLE EMBODIMENTS

Example embodiments of the present invention will now be described indetail with reference to the drawings. It should be noted that therelative arrangement of the components, the numerical expressions andnumerical values set forth in these example embodiments do not limit thescope of the present invention unless it is specifically statedotherwise.

First Example Embodiment

A monitoring system 100 according to the first example embodiment of thepresent invention will be described with reference to FIG. 1. Themonitoring system 100 is a system that discovers an abnormality behind ashielding object by using, as a trigger, the fact that a target objectentering a blind spot does not appear from the blind spot even after apredetermined time elapses.

As shown in FIG. 1, the monitoring system 100 includes a video acquirer101, a detector 103, and a notifier 104. The notifier 104 has a firsttime 141. The video acquirer 101 acquires a video.

The detector 103 detects entering of a target object into a blind spotgenerated by a shielding object in the video and the appearance of thetarget object from the blind spot. If the target object does not appearfrom the blind spot even after the first time 141 elapses since enteringof the target object into the blind spot, the notifier 104 notifies amonitor 130 or the like of it.

According to this example embodiment, it is possible to discover anabnormality behind the shielding object.

Second Example Embodiment

A monitoring system according to the second example embodiment of thepresent invention will be described next with reference to FIGS. 2 to 6.FIG. 2 is a view for explaining an outline of the operation of themonitoring system according to this example embodiment. As shown in FIG.2, a target object 210 moves at a speed V from left to right in FIG. 2.There exists a shielding object 220 at the moving destination of thetarget object 210, and the presence of the shielding object 220generates a blind spot behind the shielding object 220. After that, ifthe target object 210 enters the blind spot generated by the shieldingobject 220 and becomes invisible, the target object 210 cannot betracked.

A monitoring system 200 predicts, based on the speed (V) when the targetobject 210 enters the blind spot, the timing at which the target object210 appears again. If the target object 210 does not appear at thepredicted timing or even after a predetermined time elapses since thepredetermined timing, the monitoring system 200 alerts a monitor or thelike.

For example, assume that the predicted appearance timing comes three seclater. In this case, if the target object 210 does not appear again evenafter a lapse of [3 sec+αsec], it is determined that an abnormality hasoccurred in the blind spot behind the shielding object 220, and amonitor or the like is alerted. To the contrary, if the target object210 appears within [3 sec+αsec], the monitoring system 200 determinesthat no abnormality has occurred behind the shielding object 220, anddoes not alert the monitor or the like. This can readily discover anevent such as a fall of a person or an accident or an abnormality suchas a criminal act including a drug deal behind the shielding object 220.

FIG. 3 is a block diagram showing the arrangement of the monitoringsystem 200 according to this example embodiment. The monitoring system200 includes a video acquirer 301, a determiner 302, a speed detector303, a predictor 304, a detector 305, and a notifier 306.

The video acquirer 301 acquires a video from a camera 320. Note that thenumber of cameras 320 is not limited to one, and a plurality of cameras320 may exist. The determiner 302 determines that a blind spot generatedby a shielding object exists in the video acquired by the video acquirer301. The determiner 302 may determine the presence of the blind spot byperforming two-dimensional analysis using a video of one camera orthree-dimensional analysis using videos of a plurality of cameras. Iftwo-dimensional analysis is performed, the determiner 302 may determinethat the blind spot exists in a region occupied by the shielding objectin an image.

The speed detector 303 detects an entering speed when the target objectenters the blind spot generated by the shielding object. Based on theentering speed of the target object detected by the speed detector 303,the predictor 304 predicts the appearance timing at which the targetobject appears from the blind spot. The detector 305 detects theappearance of the target object from the blind spot. If the targetobject does not appear from the blind spot even after a predeterminedtime elapses since the appearance timing predicted by the predictor 304,the notifier 306 notifies a monitor 330 or the like of it.

FIG. 4A is a table showing an example of the structure of a predictiontable 401 provided in the monitoring system 200 according to thisexample embodiment. The prediction table 401 stores predicted appearancetime and a predetermined time as an elapsed time since the predictedappearance time in association with each combination of a target objectspeed and a blind spot size. With respect to the appearance timing ofthe target object, the predictor 304 may detect the size of theshielding object, and derive the appearance timing from the enteringspeed of the target object and the detected size of the shieldingobject. The predictor 304 may specify the appearance timing using theprediction table 401 shown in FIG. 4A.

FIG. 4B is a table showing an example of the structure of a shieldingobject table 402 provided in the monitoring system 200 according to thisexample embodiment. The shielding object table 402 stores the positionof the blind spot generated by the shielding object in association withthe position and size of the shielding object. With respect to the blindspot generated by the shielding object, the monitoring system 200 maydetect the size of the shielding object and derive the position of theblind spot based on the detected size. Furthermore, the monitoringsystem 200 may specify the position of the blind spot using theshielding object table 402 shown in FIG. 4B.

FIG. 4C is a table showing an example of the arrangement of anotification table 403 provided in the monitoring system 200 accordingto this example embodiment. The notification table 403 stores an alertlevel and a notification destination in association with an appearancedelay time. For example, if a delay time is “T4”, the alert level is “1”and the notification destination is “monitor”. As an alert, for example,the alert level may be changed in accordance with an elapsed time(appearance delay time) since the predicted appearance timing. Forexample, the alert level may be changed by gradually increasing thevolume of an alarm sound, flickering display on a screen viewed by themonitor, or gradually increasing the display.

FIG. 5 is a view showing the hardware arrangement of the monitoringsystem 200. The monitoring system 200 includes a CPU (Central ProcessingUnit) 501, a ROM (Read Only Memory) 502, a RAM (Random Access Memory)503, a storage 504, and a communication controller 505.

The CPU 501 is an arithmetic processing processor, and implements thefunctional components of the monitoring system 200 by executing aprogram. Note that the number of CPUs 501 is not limited to one, and aplurality of CPUs or a GPU (Graphics Processing Unit) for imageprocessing may be included. The ROM 502 is a read only memory, andstores programs such as firmware.

The communication controller 505 communicates with another apparatus viaa network. The communication controller 505 may include a CPUindependent of the CPU 501, and write or read out transmission/receptiondata in or from the RAM 503.

The RAM 503 is a random access memory used as a temporary storage workarea by the CPU 501. An area to store data necessary for implementationof the example embodiment is allocated to the RAM 503. The monitoringsystem 200 temporarily saves, as the data, predicted appearance time531, a delay time 532, notification contents 533, and shielding objectinformation 534. The RAM 503 includes an application execution area 535for executing various application modules.

The storage 504 is a storage device that stores programs and databasesnecessary for implementation of this example embodiment. The storage 504stores the prediction table 401, the shielding object table 402, thenotification table 403, a monitoring program 541, and a control program545.

The monitoring program 541 stores a determination module 542, a speeddetection module 543, and a prediction module 544. These modules 542 to544 are read out by the CPU 501 into the application execution area 535,and executed. The control program 545 is a program for controlling thewhole monitoring system 200. It is desirable to provide a DMAC (DirectMemory Access Controller) for transferring data between the RAM 503 andthe storage 504 (not shown).

Note that programs and data which are associated with thegeneral-purpose functions of the monitoring system 200 and otherfeasible functions are not shown in the RAM 503 or the storage 504 ofFIG. 5. The above-described hardware arrangement of the monitoringsystem 200 is merely an example. The present invention is not limited tothis, and various hardware arrangements can be adopted.

FIG. 6 is a flowchart for explaining the processing procedure of themonitoring system 200. In step S601, the monitoring system 200 acquiresa video from a camera. In step S603, the monitoring system 200recognizes the presence of a shielding object in the acquired video, anddetermines the presence of a blind spot generated by the shieldingobject. In step S605, the monitoring system 200 detects the enteringspeed of a target object entering the blind spot behind the shieldingobject, and predicts the appearance timing of the target object from theblind spot. In step S607, the monitoring system 200 determines whetherthe target object appears from the blind spot at the predictedappearance timing (predicted appearance time) or within a predeterminedtime since the appearance timing. If the target object does not appearfrom the blind spot at the predicted appearance timing or within thepredetermined time since the appearance timing, the monitoring system200 gives an alert in step S609.

According to this example embodiment, since the appearance timing of thetarget object is predicted, it is possible to discover an abnormalitysuch as a fall of a person, an accident, or a criminal act including adrug deal behind the shielding object.

Third Example Embodiment

A monitoring system 700 according to the third example embodiment of thepresent invention will be described next with reference to FIGS. 7 to 9.FIG. 7 is a view for explaining an outline of the operation of themonitoring system 700 according to this example embodiment. Themonitoring system 700 according to this example embodiment is differentfrom the above-described second example embodiment in that a directiondetector is provided. The remaining components and operations are thesame as those in the second example embodiment. Hence, the samereference numerals denote the same components and operations, and adetailed description thereof will be omitted.

The monitoring system 700 detects the entering direction of a targetobject 210 into a blind spot generated by a shielding object 220. Basedon the detected entering direction and speed, the appearance timing andappearance position of the target object 210 are predicted.

FIG. 8 is a block diagram showing the arrangement of the monitoringsystem 700. The monitoring system 700 further includes a directiondetector 801. The direction detector 801 detects the entering directionof the target object into the blind spot generated by the shieldingobject. Based on the entering direction and the entering speed, themonitoring system 700 predicts the appearance position of the targetobject together with the appearance timing of the target object.

FIG. 9 is a flowchart for explaining the processing procedure of themonitoring system 700. Note that the same step numbers as those in FIG.6 denote the same steps, and a description thereof will be omitted. Instep S901, the monitoring system 700 further detects the enteringdirection of the target object into the blind spot, and predicts theappearance position of the target object from the blind spot based onthe detected entering direction. In step S903, the monitoring system 700determines whether the target object appears at the predicted appearanceposition at the predicted appearance timing.

According to this example embodiment, since the appearance position ispredicted by additionally considering the entering direction of thetarget object into the blind spot generated by the shielding object, itis possible to discover an abnormality such as a fall of a person, anaccident, or a criminal act including a drug deal behind the shieldingobject.

Fourth Example Embodiment

A monitoring system 1000 according to the fourth example embodiment ofthe present invention will be described next with reference to FIGS. 10to 12. FIG. 10 is a view for explaining an outline of the operation ofthe monitoring system 1000 according to this example embodiment. Themonitoring system 1000 according to this example embodiment is differentfrom the above-described third example embodiment in that a videogenerator is provided. The remaining components and operations are thesame as those in the third example embodiment. Hence, the same referencenumerals denote the same components and operations, and a detaileddescription thereof will be omitted.

If a target object 1010 enters a blind spot generated by a shieldingobject 1020, the monitoring system 1000 displays, based on an enteringspeed at which the target object 1010 enters the blind spot, a predictedposition mark 1011 indicating a predicted position at which the targetobject 1010 is located in the blind spot generated by the shieldingobject 1020. Furthermore, the monitoring system 1000 displays apredicted moving trajectory 1030 together with the predicted positionmark 1011. The monitoring system 1000 may display the predicted positionmark 1011 and the predicted moving trajectory 1030 in consideration ofan entering direction in addition to the entering speed of the targetobject 1010 into the blind spot. The monitoring system 1000 may alsopredict the appearance position of the target object 1010 entering theblind spot from the blind spot, and display the predicted appearanceposition. For example, the monitoring system 1000 may display theappearance position like a target object 210 indicated by dotted linesin FIG. 2 or a target object 210 indicated by dotted lines in FIG. 7 (ora target object 1310 indicated by dotted lines in FIG. 13 to bedescribed later), or display the appearance position as a region.

FIG. 11 is a block diagram showing the arrangement of the monitoringsystem 1000. The monitoring system 1000 further includes a videogenerator 1101. The video generator 1101 generates a video bysuperimposing, on a video acquired by a video acquirer 301, thetrajectory of the target object generated based on prediction by apredictor 304. The video generator 1101 shows the generated video to amonitor 1130 or the like.

FIG. 12 is a flowchart for explaining the processing procedure of themonitoring system 1000. Note that the same step numbers as those in FIG.6 denote the same steps, and a description thereof will be omitted. Instep S1201, the monitoring system 1000 generates a video bysuperimposing, on an acquired video, the moving trajectory of the targetobject generated based on prediction. The monitoring system 1000 showsthe generated superimposed video to the monitor 1130 or the like.

According to this example embodiment, since the predicted position markand the predicted moving trajectory are displayed, it is possible todiscover an abnormality such as a fall of a person, an accident, or acriminal act including a drug deal behind the shielding object, and themonitor can visually recognize the appearance timing of the targetobject.

Fifth Example Embodiment

A monitoring system 1300 according to the fifth example embodiment ofthe present invention will be described next with reference to FIGS. 13to 15. FIG. 13 is a view for explaining an outline of the operation ofthe monitoring system 1300 according to this example embodiment. Themonitoring system 1300 according to this example embodiment is differentfrom the above-described third example embodiment in that a predictorconsiders a relative speed. The remaining components and operations arethe same as those in the second example embodiment. Hence, the samereference numerals denote the same components and operations, and adetailed description thereof will be omitted.

FIG. 13 is a view for explaining an outline of the operation of themonitoring system 1300 while a target object 1310 and a shielding object1320 move. The target object 1310 moves at a speed V1 from left to rightin FIG. 13. A vehicle such as a bus serving as the shielding object 1320moves at a speed V2 from right to left in FIG. 13. In this case, themonitoring system 1300 predicts the appearance timing of the targetobject 1310 based on the relative speed between the target object 1310and the shielding object 1320.

FIG. 14 is a block diagram showing the arrangement of the monitoringsystem 1300. The monitoring system 1300 includes a relative speedcalculator 1401 and a predictor 1402. The relative speed calculator 1401calculates the relative speed between the moving target object 1310 andthe moving shielding object 1320. The predictor 1402 predicts theappearance timing of the target object 1310 based on the relative speedcalculated by the relative speed calculator 1401.

FIG. 15 is a flowchart for explaining the processing procedure of themonitoring system 1300. Note that the same step numbers as those in FIG.6 denote the same steps, and a description thereof will be omitted. Instep S1501, the monitoring system 1300 calculates the relative speedbetween the moving target object 1310 and the moving shielding object1320. In step S1503, the monitoring system 1300 predicts the appearancetiming of the target object 1310 from the blind spot based on thecalculated relative speed.

According to this example embodiment, since prediction is performed inconsideration of the relative speed, it is possible to discover anabnormality such as a fall of a person, an accident, or a criminal actincluding a drug deal behind the shielding object.

Other Example Embodiments

While the invention has been particularly shown and described withreference to example embodiments thereof, the invention is not limitedto these example embodiments. It will be understood by those of ordinaryskill in the art that various changes in form and details may be madetherein without departing from the spirit and scope of the presentinvention as defined by the claims.

The present invention is applicable to a system including a plurality ofdevices or a single apparatus. The present invention is also applicableeven when an information processing program for implementing thefunctions of example embodiments is supplied to the system or apparatusdirectly or from a remote site. Hence, the present invention alsoincorporates the program installed in a computer to implement thefunctions of the present invention by the computer, a medium storing theprogram, and a WWW (World Wide Web) server that causes a user todownload the program. Especially, the present invention incorporates atleast a non-transitory computer readable medium storing a program thatcauses a computer to execute processing steps included in theabove-described example embodiments.

1. A monitoring system comprising: at least one memory storinginstructions; and at least one processor coupled to the at least onememory and configured to execute the instructions to: acquire a video;detect an entering of a target object into a blind spot generated by ashielding object in the video; detect an entering speed of the targetobject entering the blind spot; estimate a position of the target objectin the blind spot based on the entering speed; estimate a path of thetarget object behind the shielding object; cause a display device todisplay the acquired video; and cause the display device to display, onthe acquired video, marks indicating the estimated position of thetarget object in the blind spot and the estimated path of the targetobject behind the shielding object.
 2. The monitoring system accordingto claim 1, wherein the at least one processor is further configured to:detect an entering direction of the target object into the blind spot;and estimate the position of the target object based on the enteringspeed and the entering direction.
 3. The monitoring system according toclaim 1, wherein the marks include an arrow linking a position where thetarget object entered the blind spot and a predicted appearance positionof the target object from the blind spot.
 4. The monitoring systemaccording to claim 2, wherein the at least one processor is furtherconfigured to: predict an appearance position of the target object fromthe blind spot based on the entering direction; and cause the displaydevice to further display the predicted appearance position.
 5. Amonitoring method comprising: acquiring a video; detecting an enteringof a target object into a blind spot generated by a shielding object inthe video; detecting an entering speed of the target object entering theblind spot; estimating a position of the target object in the blind spotbased on the entering speed; estimating a path of the target objectbehind the shielding object; causing a display device to display theacquired video; and causing the display device to display, on theacquired video, marks indicating the estimated position of the targetobject in the blind spot and the estimated path of the target objectbehind the shielding object.
 6. The monitoring method according to claim5, further comprising: detecting an entering direction of the targetobject into the blind spot; and estimating the position of the targetobject based on the entering speed and the entering direction.
 7. Themonitoring method according to claim 5, wherein the marks include anarrow linking a position where the target object entered and anpredicted appearance position of the target object from the blind spot.8. The monitoring method according to claim 6, further comprising:predicting an appearance position of the target object from the blindspot based on the entering direction; and causing the display device tofurther display the predicted appearance position.
 9. A non-transitorycomputer-readable storage medium storing a program that causes acomputer to perform: acquiring a video; detecting entering of a targetobject into a blind spot generated by a shielding object in the video;detecting an entering speed of the target object entering the blindspot; estimating a position of the target object in the blind spot basedon the entering speed; estimating a path of the target object behind theshielding object; causing a display device to display the acquiredvideo; and causing the display device to display, on the acquired video,marks indicating the estimated position of the target object in theblind spot and the estimated path of the target object behind theshielding object.
 10. The storage medium according to claim 9, whereinthe program further causes the computer to perform: detecting anentering direction of the target object into the blind spot; andestimating the position of the target object based on the entering speedand the entering direction.
 11. The storage medium according to claim 9,wherein the marks include an arrow linking a position where the targetobject entered and a predicted appearance position of the target objectfrom the blind spot.
 12. The storage medium according to claim 10,wherein the program further causes the computer to perform: predictingan appearance position of the target object from the blind spot based onthe entering direction; and causing the display device to furtherdisplay the predicted appearance position.